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Abstract 

0^ ' This paper has a two- folded purpose. First, we attempt to out- 

^^ ■ Une the development of the turnpike theorems in the the last several 

decades. Second, we study turnpike theorems in finite-horizon two- 

Pm [ person zero-sum Markov games on a general Borel state space. Utilis- 

ing the Bellman (or Shapley) operator defined for this game, we prove 

rG , the stochastic versions of the early turnpike theorem on the set of op- 

Cd I timal strategies and the middle turnpike theorem on the distribution 

of the state space. 
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O '• 1 Introduction 

(N, 

Turnpike theorems are a set of economic theorems on optimal control prob- 
lems, which characterise solutions (optimal paths) of optimal control prob- 
^ ! lems and focus on the stationary optimal one. The turnpike theorem was 

j^ I first proposed by Dorfman, Samuelson and Solow in [2^ within the context 

of an optimal growth model. In the literature, there are extensive work on 
the turnpike theorems in many diversified settings of optimal control prob- 
lems and games, ranging from discrete time to continuous time, from the 
deterministic case to the stochastic case, with or without discounting. There 
is no clear-cut and unified classification for turnpike theorems. For exam- 
ple, McKenzie [67] distinguished three kinds of turnpikes: the early, late and 
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middle turnpikes, but in [7T], he used a different classification scheme with 
the Samuelson and Ramsey turnpike. We think that this is helpful to clar- 
ify different types of turnpike theorems and to give a clear picture on the 
state of this subject. Hence we start this paper with a survey on the main 
developments of the turnpike theorems in the last several decades. 

Further on, in this paper, we study turnpike theorems in finite-horizon 
zero-sum Markov games on a general Borel state space following the approach 
developed in [50] in which a finite state space was considered. A general 
approach to Borel state space stochastic games was developed by Maitra 
and Sudderth [65] . They proved the existence of the game value for general 
universally measurable strategies. However, the optimal strategies need not 
exist in their games and it is assumed the transition probability function is 
strongly continuous in the actions of one player. 

In our setting, under the weaker assumption (14. ip in Section HI we show 
the convergence of the long time average cost (reward) of the game and prove 
the turnpike theorems on the set of strategies and on the distribution of the 
states. 

The organisation of this paper is as follows. In Section HI we sketch a 
picture of turnpike theory showing how the theory was developed. In Section 
|3l we present the basic setting for a two-person zero-sum Markov game and 
define the Bellman (or Shapley) operator. The main results will be presented 
and proved in Section HI Section O concludes this paper and proposes further 
research. 



2 Recent development in the turnpike theory 

In this section, we outline the development of the turnpike theory in the last 
several decades. We shall start with turnpike theorems in the deterministic 
setting, followed by the stochastic analog of the turnpike theorems. 

2.1 The turnpike theorems in the deterministic setting 

For the turnpike theorems in the deterministic setting, it is important to dis- 
tinguish the asymptotic and neighborhood turnpike theorems for solutions to 
discounted infinite-horizon optimal control problems from the classical turn- 
pike theorems for solutions to undiscounted finite-horizon problems. Some 
surveys in the deterministic setting can be also found in [TH IHH El SB]- 



The classical turnpike theorem considers a deterministic finite-horizon 
(T > 0) optimal control problem on a given state space X. In a discrete 
time case, the problem is 

{maximise '^t=o -^(^ti^t+i) 

subject to Xq.Xt G X given, {xt}J=Q C X 

with a cost (utility) function L : X"^ -^ R, T eN. In a continuous time case, 
the problem is 

maximise J^ L{xt,Ut)dt 

subject to Xt = h{xt,Ut), Ut E U (2-2) 

Xo,Xt e Xgiven, {xt}te[o,T] C X 

with a cost (utility) function L : X x U ^ R, where h : X x U ^ R 
is a continuous mapping with respect to both arguments, U is the set of 
admissible controls and T G M"'". The initial value Xq and the terminal value 
xt are held fixed. Let x* denote a solution of the maximization problem 

maximise L{x,x), x E X. 

The path x* := {xt : Xt = x*,\/t E [0, T]} is called a stationary optimal path. 
Often, the uniqueness of the stationary optimal path is assumed. 

In the case where the terminal value xt is not given, an additional term, 
interpreted as the terminal payoff function, should be added to the above 
problems. That is, the optimal problem is, in a discrete time setting 



[maximise X]i=o ^i^t,Xt+i) + S{xt) 



(2.3) 



[subject to xo given, {xt}f=Q C X 
and, in a continuous time case 

{maximise J^ L{xt,Ut)dt + S{xt) 

subject to Xt = h{xt,Ut), Ut E U (2-4) 

Xo given, {xt}te[o,T] C X 

where the terminal payoff function 5" : X — )> R. 



The original formulation of the continuous time problem f l2.2p is ascribed 
to Samuelson-Solow [S2] and the discrete time problem (12.1 p to Gale [32] and 
Mckenzie [78l|67]. It was further investigated by e.g. Radner |80], Morishima 
|77j . McKenzie [66] and Makarov and Rubinov [61], Atsumi [1] and Haurie 
[35] . Mckenzie [71] discussed in detail the development of the turnpike the- 
ory in both the von Neumann growth model and the Ramsey's growth model 
and specified a classification of Samuelson turnpike and Ramsey turnpike. 
The classical techniques which were developed to prove the classical turn- 
pike theorems are critically dependent on the convexity of the control set 
U and technology function h and the preference function L. These classi- 
cal techniques fall into two classes: Value-Loss methods and Monotonicity 
method, see more discussion in [IS]. 

The classical turnpike theorem states that if T is large enough, the optimal 
path {xl} that transfers the system from xq to xt approaches the unique 
optimal stationary level x*, and stays close to it for a large fraction of T, and 
moves away toward the terminal state only in the final periods; that is, for 
each e > 0, there exist an integer Tq such that for each T > 2Tq, 

\\x;-x*\\<e, \/te[To,T-To]. (2.5) 

This is called the middle turnpike theorem in [HZ]. Moreover, if this result 
holds for any initial value Xq G X, it is the global version; if the result holds 
for some initial value xq which is near enough to x*, it is the local version, e.g. 
[9]. Further, if for each e > 0, there exists an integer Tq and a number 6 > 
such that \\xo- x*\\ < 6 implies ([23]) for all t e [0,T - Tq] with T > 2Tq, 
this was called the early turnpike theorem in [57] . Since this image resembles 
a map of an intercity highway or turnpike with entrance and exit ramps, a 
stationary optimal path x* is often referred to as a turnpike. 

A version of the classical turnpike theorem for games was also presented. 
For example, two-player zero-sum differential games were studied by Za- 
slavski [HI] , showing that there exists a pair of optimal stationary paths x* 
and y* for the two players and presenting a version of classical middle turn- 
pike theorem, and by Alvarez and Bardi [3l|2l|8], proving a turnpike theorem 
for the value of the game. 

In the classical setting, the state space X was often a compact, convex, 
bounded and closed subset of a finite-dimensional space; the cost (utihty) 
function L is often continuous, smooth and strictly concave (or convex); 
hence the turnpike theorem was proved by the concavity analysis. Later, 



under upper (lower) seniicontinuity, some extensions to general Banach state 
spaces were obtained by e.g. Kolokoltsov ^U\, Yakovenko and L.A. Kontorer 
[88j, Zaslavski [901 |92l |93] , Mamedov [59]. 

When the future is discounted, the deterministic optimal control problem 
with infinite-horizon is described, in a discrete-time setting as 



{maximise Xlt^o^*-^(^t'^t+i) 

subject to Xq G X, {xtj'^Q C X 



(2.6) 



and in a continuous-time setting as 



{maximise /o°° ^ '^*L{xt,Ut)dt 

subject to Xt = h{xt,ut), Ut G U (2-') 

xo G X, {xt}t>o C X 

with a discount factor < 6 < 1 and a discount rate p > 0. The case 
where 6 = 1 and p = is referred to as the undiscounted version of the 
problem (12. 6 p and (12. 7p . The asymptotic behaviour of the optimal path in 
the discounted framework depends critically on the magnitude of the initial 
stock and the discount factor. 

Two classes of turnpike theorems should be distinguished when the future 
is discounted. The first class is the asymptotic turnpike theorem, see e.g. in 
Scheinkman [85], Bewley [6] , Yano [87], Carlson[T5], Carlson, Haurie and 
Jabrane [13], Zaslavski [89], Park [79]. It states that there exists a discount 
factor S' G (0, 1) such that for Xq G X and any 6 G [§', 1), the optimal path 
{x*} starting at Xq converges to the stationary path x*. 

The asymptotic turnpike theorem was proved under strictly concavity 
assumptions on L by e.g. Scheinkman [85] and Bewley [6], Majumdar, Ner- 
muth [63], Kamihigashi and Roy [17]. Majumdar and Mitra [61] interpreted 
the function L as the consumption and set L{xt, Xt+i) := f{xt) —Xt-^-i > for 
all t and considered a nonconvex (convex-concave) technology function /, i.e. 
/ is assumed to be strictly increasing and twice continuously differentiable 
such that / satisfies f'{oo) < 1 < /'(O) < oo and there is a real number 
ki > such that f"{x) = for x = ki, f"{x) > for < x < A;i and 
f"{x) < for X > ki. 

Montrucchio [721 [75] assumed certain curvature restrictions, i.e. the func- 
tion L is continuous and strongly (a, /3)-concave, i.e. L{x,y) + |a|xp -|- 



|/?|yp is concave with a + /3 > and that the value function W{xo) := 
niax^^Q (5*L(a;t, Xf+i) is concave-7 for all 6 G {60, 1), i.e. W{x) + |7|a;p is 
convex for all 6 G (Sq,!). He proved that there exists 6' = 1 — ^-^ such that 

for a discount factor 5 G (max[5o) 1 ~ ^-^]) 1) the local asymptotic turnpike 
theorem holds, where a + /3 is a measure of the lower curvature of L and 7 
is a measure of its upper curvature. Carlso and Haurie [16] proved a version 
of the asymptotic turnpike theorem for infinite horizon open-loop differen- 
tial games with 5 = 1 under a "strict diagonal concavity condition" on L. 
An uniform asymptotic turnpike theorem was proved by Cuong LeVan and 
Lisa Morhaim [58j under similar assumptions, i.e. if the function L{x^ y) is 
a strictly concave function, increasing in x and decreasing in y, then for any 
initial point Xq G X, there exists a /?' G (0, 1) such that for any /3 G [13', 1), 
the asymptotic turnpike property holds. 

Araujo and Scheinkman [S], McKenzie |68|, Dasgupta and Mckenzie [2T] 
and Hyun Park ^79j applied the implicit function theorem to prove the asymp- 
totic turnpike theorem by assuming a dominant diagonal blocks condition: 
for an infinite matrix D formed by n x n blocks Dij, i,j G {1,2,...}, Da is 
invertible, supj \D~-^\ < 00 and supj^°^^ ^ ,• iD^^-^Dijl < 1, where the norm 
lAjl = sup|^|=i|Aj2;|. 

The second class is the neighborhood turnpike theorem, see e.g. |T3l [691 
EH [m [701 |28l |62l HZ]. It states that for any e > 0, there exists T' > and 
S' G (0, 1) such that, an optimal path stating from xq at the discount factor 
6 G {S',1) eventually stays within the e- neighborhood of a stationary path 
X*, i.e. 

\\x*-x*\\<e, yt>T'. 



't 



If a smaller neighborhood e is chosen, the closer the discount factor 5 is to 1. 

To prove the neighborhood turnpike theorem, e.g. in Mckenzie [70], Yano 
[87] , instead of strictly concavity assumption, the differentiability of the func- 
tion L is required. A version of uniform neighborhood turnpike theorem was 
proved in Guerrero-Luchtenberg and Leonardo [ST]. It states that, under 
certain differentiability and concavity conditions on L, the neighborhood 
turnpike theorem holds for almost all initial states. 

Finite-horizon problem with discounting was examined in e.g. [301 128] . 
Based on convexity condition and using Implicit Programming formulation, 
Feinstein and Oren [30] proved a funnel asymptotic turnpike theorem, namely, 
an optimal path lies in an exponentially bounded region (determined by the 
discount factor) of the optimal stationary path. Fershtman and Kamien [28] 



considered a finite-horizon differential games with quadratic cost function 
and proved a neighborhood turnpike theorem for the feedback equilibrium 

strategies. 

2.2 The stochastic version of turnpike theorems 

The turnpike theorems under uncertainty was first developed by William and 
Mirman [TO]. The uncertainty or the stochastic environment is represented 
by a filtered probability space {Q,J^,iy; {J^t}t>o), u E fl is a possible state 
of the environment, J-" is the Borel a- field, z/ is a probability measure on 
Q, {J^t}t>o is the filtration satisfying the "usual" conditions, i.e. it is right- 
continuous and complete. The underlying evolution of the states {xf} is a 
stochastic process on [Q, J-", z/). Then the stochastic optimal control problem, 
in the undiscounted case, can be described, in the discrete-time setting as 



maximise E 



^L{xt,ut) 
t=o 



(2.8) 



and in the continuous-time setting as 



maximise 



E 



L{xt,ut)dt 



IJO 



(2.9) 



subject to the constraint {xt, Ut) E X x U with the initial condition xq E X 
and the terminal condition x\t=T = xt E X with probability 1. Here, in 
general, Xq is an J-'o-measurable random variable and xt is an J-j^-measurable 
random variable. An optimal path (x*, -u*) is an J^^-adapted process such that 
(EU or dMD is achieved. 

In the case where the terminal value is not given and a terminal payoff 
function S is considered, the problems can be reformulated as 



maximise E 



^L{xt,ut) + S{xt) 



t=o 



(2.10) 



and 



maximise E 



/ L{xt,ut)dt + S{xt) 
Jo 



(2.11) 



Further, when the future is discounted, the problem is described, in the 
discrete-time setting as 



maximise E 



■ T 
t=0 



(2.12) 



and in the continuous-time setting as 



maximise E 



e ^^L{xt,ut)dt 



(2.13) 



The stochastic analog of the turnpike theorem states that the optimal 
paths converge in a stochastic sense (say, convergence in distribution, con- 
vergence in probability or almost surely convergence) to a unique stochastic 
limit under standard assumptions of differentiability and concavity of the 
function L. The proof was mostly based on dynamic programming principle. 

The most studied stochastic version of the middle turnpike theorem states 
that for any e > 0, there exists a number 6 > {) such that the expected number 
of time periods for which {x^ } spends outside the e-neighbourhood of x*, i.e. 
Ilx* — x*|| > e, is bounded from above by ^, where 9 is independent of T. 

The uniformity assumption and the sensitivity assumption were mostly 
used to prove the stochastic version of turnpike theorems, as discussed in 
Joshi |46j. The idea is first to construct a value-loss process, say {Vt : t > 0} 
which has the convenient property of being either a martingale (Majumdar 
and Zilcha [76]), a submartingale (Brock and Majumdar [12] or a super- 
martingale (Joshi [35]) thereby permitting a passage to the rich theory of 
martingales. Then the process {Vj : t > 0} is assumed to be uniform bounded 
in expectation (i.e. the uniformity assumption) and have certain sensitivity 
property (i.e. the sensitivity assumption). The sensitivity assumption of 
{Vt : t > 0} says that, for any e > 0, if the optimal paths diverge from the 
stochastic limit by more than e, then the process {Vt} records a value- loss of 
at least 77(e) > 0. In other words, value-loss should be sensitive to a critical 
divergence of the optimal paths. 

For general Markov chains the question on the existence of optimal value 
for discounted prices wwas addressed in [38]. Turnpike theorems for Markov 
chains were proved by Shapiro [H3] in discounted case and by Kolokoltsov 
[39] without discounting. In the latter paper the turnpike theorems on the 
strategies and on the distributions of states were distinguished. 
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For the financial portfolio problems, the turnpike theorems was proved, 
mainly under convexity conditions on the function L directly, e.g. by Mossin 
[78], Leland |56], Hakansson [35], Hakansson |3l], Sumit Joshi j35], Cox and 
Huang [19] and Huang and Zariphopoulou [IQ], Dybvig, Rogers and Back 
Kerry [23]. In the discrete-time setting, Mossin [78] showed the convergence 
of optimal strategies under the assumption that — ^}^\ = ax + b, where L' 
and L" is the first and second order derivative of L, respectively. Leland 
[56] generalised Mossin's result by assuming that —777© = ax + h{b) where 



L"(x) 

the function h is uniformly bounded. Hakansson [33] proved convergence of 



optimal strategies by making assumptions slightly more general ^ ^^ — < 

L{x) < ~^"_h — !^^ ^ ^- Huberman and Ross [41j showed that convergence 
occurs for the functions L that are bounded from below and for some c G 
(0 1) lim -L"{x)x _ 

In the continuous-time setting. Cox and Huang [19] demonstrate that the 
portfolio turnpike property holds if the utility function L is strictly concave 
with lim^^oo L' (x) = and there exists Ai,A2,b > and K > such 
that \{L')~^{x) — Aix~b\ < A2X~"-, for all x < K and some a € [0,-^), 
where {L')~^ denotes the inverse of L'. Huang and Zariphopoulou [30] gave 
another sufficient condition for the turnpike property that the strictly concave 
function L satisfies the condition lim2;_5.oo —f^ = k for < r < 1 and k > 0. 

In optimal growth theory, mainly under the uniformity assumption and 
the sensitivity assumption, in the undiscounted case. Brock and Mirman 
jllj . Dana [20], Evstigneev |25[ [26] |3] and Mirman and Zilcha [75] showed 
that all optimal paths converge in an appropriate topology to the optimal 
stationary path. In the discounted context. Brock and Mirman [10] and 
Mirman and Zilcha [73] proved the convergence in distribution of optimal 
paths from distinct initial stocks to the stationary optimal paths. Fleming, 
Sethi and Soner ^31j| studied production planning problem where the demand 
is described as a Markov chain and showed a turnpike theorem for the Markov 
control. 

Some research considered a so-called non- stationary setting. The stochas- 
tic environment can be non-stationary, namely, Q := X^qQi and is consid- 
ered as the set of all sequences u := {ut : oJt G Qt,t > 0}, where Ut is a 
possible state of the environment at time t. If the function L depends addi- 
tionally on time t, it is called time-varying or non-stationary preference. In 
non-stationary models, the convergence in probability (Brock and Majum- 
dar p^) and the stronger property of almost sure convergence (Chang [T7] . 



Follmer and Majumda [22], Joshi [IS], Majumdar and Zilcha [7S], Joshi [15] ) 
were proved. Denardo and Rothblum [22] considered an exponential utility 
function L for a Markov decision process and proved that if the corresponding 
transition rate matrix is transient, there exits an optimal stationary control 
path u*. 

In game theory, turnpike theorems for long-time two-player zero-sum 
stochastic games on a finite state space were proved by e.g. Kolokoltsov 
[50\ . Kolokoltsov and Malafeyev[51j. by utilising the Bellman (or Shapley) 
operator and imposing conditions on the transition probability. Related re- 
search on long-time two-player zero-sum stochastic games can be founded e.g. 
in Sorin [H3], Hernandez- Lerma and Lasserre (33 , Guo and Hernndez-Lerma 



Haurie and Delft [36] considered a dynamics games with piecewise deter- 
ministic path and random jumps. In the discounted setting, under the as- 
sumption of convexity and piecewise uniqueness, the global asymptotic was 
proved if the transition probability qij are constant and the local asymptotic 
was proved if q^j are state and control dependent. 

For zero-sum Markov games, Jaskiewicz [12], Vega-Amaya [H^ worked 
with strongly continuous transition probabilities, i.e. the transition prob- 
ability q{D\x,u,v) is continuous in actions u and v, for any fixed states x 
and Borel subset D of the state space X. It was proved that the optimality 
equation has a solution, there exists a value of the game and both players 
possess optimal stationary strategies. In the proof, Jaskiewicz [32] analyzes 
auxiliary perturbed models, whereas Vega-Amaya ^86j makes use of a fixed 
point theorem. 

In the work of Jaskiewicz and Nowak [H] and Kiienle [55], zero-sum 
Markov games within the similar setting where the transition times occur at 
integer time moments were studied. Jaskiewicz and Nowak [33] prove the 
similar results by applying Fatou's Lemma for weakly convergent measures. 
Kiienle [55] introduced certain contraction operators which lead to a param- 
eterized family of functional equations. Making use of some continuity and 
monotonicity properties of the solutions to these equations (with respect to 
the parameter) he obtains a lower semicontinuous solution of the optimal- 
ity equation. Then he showed that the maximzing player has an e-optimal 
stationary strategy, and the minimizing player has an optimal stationary 
strategy. 

Jaskiewicz [13] studied a zero-sum ergodic semi-Markov games, in a gen- 
eral Borel state space, with weakly continuous (Feller) transition probabil- 
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ities and lower semicontinuous payoff function L. Under the Banach fixed 
point theorem and an ergodicity assumption of an embedded Markov chain, 
it was proved that in this game, one player possesses an e-optimal stationary 
strategy and the other has an optimal stationary strategy. 

Beyond the uniqueness assumption on the turnpike, it is worth noting that 
if there are several optimal stationary paths, there exists a state transition 
of the optimal paths from one optimal stationary path to another, see e.g. 
Mamedov [60], Alain Rapaprot and Pierre Cartigny [Slj, Kamihigashi and 
Roy [47]. 

3 Basic setting for a zero-sum Markov game 

In this section, we shall set up a model for a discrete-time two-person zero- 
sum Markov game on a general Borel state space. A Bellman (or Shap- 
ley)operator is defined for this game and some basic properties of this oper- 
ator are discussed. 

A discrete-time two-person zero-sum T-step, T E N, Markov game is a 
tuple {X, 71, U, V, P, g, S) of the following meanings: 

• X is a Borel subset of a general complete metric state space endowed 
with its Borel a-field B{X) and a Borel cr-finite reference measure n on it. 
One can always keep in mind the sets {1, • • ■ , ri} and N with arbitrary discrete 
measure or a subset of R'^ with Lebesgue measure as basic examples of X 
with its reference measure n. 

• U and V are compact spaces of the admissible strategies (i.e., control 
parameters) for player 1 and player 2, respectively. 

• P = {P{u,v,x,-),x E X,u E U,v E V} is the family of transition 
probability kernels, i.e. P{u, v,x,A), A E B{X), is the transition probability 
from state x to A if the two players choose strategies u and v respectively. We 
assume that the transition probability P{u,v,x, ■) has a density p{u,v,x,y) 
with respect to the reference measure 7r(-). 

• g : ?7xV^xXxXi— j-Misa transition cost function for player 1, namely, 
g{u, V, X, y) is the cost of the player 1 (i.e., the income of the player 2) from 
the transition from x to y under controls u and v. If the cost g{u, v, x, y) is 
negative, this means that the player 2 pays \g{u,v,x,y)\ to player 1. The 
function g is assumed to be continuous and also bounded by certain constant 
G > 0, uniformly in u, v, x, y, i.e. 

\g{u,v,x,y)\<G, \/u,v,x,y. 
11 



• 5* : X I—)- M is a final cost function depending on the final position. The 
function S is assumed to be bounded and continuous. We denote the set of 
bounded and continuous functions on X by Cb{X). 

The dynamic T-step game with a given initial position x G X is played 
as follows. At the first step, starting from the given initial state x, the 
players choose sequentially certain strategies v E V and then u E U and 
the game moves to the next state y E X with probability p{u,v,x,y)7i{dy). 
Then the player 1 pays g{u,v,x,y) to player 2 if g{u,v,x,y) is positive; 
otherwise, the player 2 pays \g{u,v,x,y)\ to player 1. The second step is 
played analogously starting from the new position y. After T steps, the 
player 2 receives additional payment S{k) depending on the finial position 
k E X. In this multi-step game, the player 1 aims to minimise the payoff to 
player 2. 

We define the Bellman operator B : Cb{X) i-)- Cb{X) by 



{BS){x) := inf sup 



X 



{g{u, V, X, y) + S{y))p{u, v, x, y)'Tc{dy) 



(3.1) 



Then, according to the dynamic programming principle, the solution to the 
T-step Markov game is given by the iterations B^S. We would like to men- 
tion that, since the transition cost function g and the final cost function S 
are assumed to be bounded and continuous, the inf sup in the definition (13. 2p 
can be replaced by minmax. Namely, in our setting, the Bellman operator 
B can be written as 



{BS){x) = minmax 



X 



{g{u, V, X, y) + S{y))p{u, v, x, y)-K{dy) 



(3.2) 



Remark 3.1. It is often assumed that the game defined by fiS.S^) has a value, 
which means that inf sup are interchangeable for all S G Cb{X) such that one 
also has 



{BS){x) = sup inf 



X 



{g{u, V, X, y) + S{y))p{u, v, x, y)n{dy) 



But we shall not make or use this assumption. Hence, in our framework, 
B^ describes the payoff for the player 1 calculated for the worst scenario 
dynamics (sometimes called robust control). 

It is clear that, for any constant a G M and any function S G Cb(X), the 
Bellman operator satisfies the following homogeneity property 

B{a + S) = a + BS. (3.3) 
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Moreover, the Bellman operator B is non- expansive in the usual sup-norm, 
i.e. 

\\BSi-BS2\\<\\Si-S2\\, \/Si,S2eCbiX) (3.4) 

since the inequality 

\\BSi-BS2\\ =sup\{BSi-BS2){x)\ 

< sup sup / \{Si{y) - S2{y))\p{u,v,x,y)7i{dy) 

xGX u.v J X 

<\\Si-S2\\ 

holds for any 81,82 G Ch{X). 

With the homogeneity property (13.31) and the non-expansive property 
(13.41) of the Bellman operator B, we can define the quotient space Cb{X) of 
Cb(X) with respect to constant functions. Namely, the equivalence classes 
are defined by 

[S] :={5 + a,VaeM: ^ G a(X)}. 

Let n : Ch{X) I— 7- Ch{X) be the natural projection. The quotient norm on 
Ch{X) is defined by the formula 

||n(/)|| = inf ||/ + a|| = \ fsup/(x) - inf /(x)) . (3.5) 

agM 2 \x(^X ^GX J 

It is clear that 11 has a unique isometric right inverse mapping $ : Cb{X) 1— >■ 
Cfe(X) so that n o $ is an identity in Ci,{X). The image ^{Cb{X)) consists 
of all functions 8 G Ci,{X) such that 

sup S'(2;) = — inf 8{x). 

Thus one can identify 8 G C\,{X) with its image 5* = $(<S) G C\y[X\ By the 
properties (13. 3p and (13.41) . the continuous quotient map B : C\XX) i-> C\XX^ 
is well defined. 

4 Main results 

In this section, we shall present the turnpike theorems for the game specified 
above. First, the uniqueness of the average cost and the stationary strategy 

13 



are proved. Then, the turnpike theorems on the set of strategies and on the 
state space are proved. 

To state the main resuhs, we need some additional properties of the 
transition probabihties: 

36>0:3Ae B{X) with7r(A) > : Vx G X, V^i G f/, Vz; G V, Vy G A, 

(4.1) 
p{u,v,x,y) > 6. 

First, we show that the operator i? is a contraction in the following 
Lemma 4.1. If the property ( [^. j| ) holds, then 

\\B{Si) - B{S2)\\ < (1 - SrriAmSi ~ ^all ^81,82 G Ck{X). (4.2) 
Proof. For arbitrary Si, S2 G Cb{X) and arbitrary states x, z E X, 
{BSi - BS2)ix) - (BSi - BS2){z) 
<max / {Si{y) - S2{y))p{u,v,x,y)'K{dy) 



X 



Denote 



-min / {Si{y) - S2{y))p{u,v,z,y)'K{dy). 

u,v J^ 



(Mi,fi) = argmax / {Si{y) - S2{y))p{u,v,x,y)'K{dy) 

u,v J^ 

and 

(M2,t'2) = argmin / {Si{y) - S2{y))p{u,v,z,y)'K{dy). 

Then, we have 

{BSi - BS2){x) - (BSi - BS2){z) 
< / {Si{y) - S2{y))yp{ui,vi,x,y) - p{u2,V2,z,y)j7v{dy) 

<||5'i-S'2|| / \p{ui,vi,x,y) - p{u2,V2,z,y)\iT{dy). 
Jx 
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(4.3) 



By property (14.1 1) , for all y G A, 

p{ui,vi,x,y)>6, and p{u2,V2, z,y) > 6. 
Then 

|p(wi, wi, X, y) - p{u2, V2, z, y)\Tc{dy) 



X 



< / ip{ui,vi,x,y) + p{u2,V2,z,y))'K{dy) 

Jx\A ^ ^ 

+ / p{ui,vi,x,y) - p{u2,V2,z,y) 7T{dy) 

J A 

=1- / p{ui,vi,x,y)'K{dy) + l- j p{u2,V2, z,y)iT{dy) 

J A J A 

+ / \p{.ui,vi,x,y) - p{u2,V2,z,y)\'K{dy). 

J A 

Denote by A^ the set oi y & A such that 

p{ui,vi,x,y) > p{u2,V2,z,y) 

and by A^ the set oi y & A such that 

p{ui,vi,x,y) < p{u2,V2,z,y). 

Clearly, A+ n A' = and A+ U A' = A, so 

|p(mi, vi, X, y) - p(w2, t;2, 2;, Z/)k(d|/) 

<2- / p{ui,vi,x,y)7r{dy)- p{u2,V2,z,y)7r{dy) 

Ja+ua- Ja+ua- 

+ / {p{u2,V2,z,y) - p{ui,vi,x,y))7r{dy) 



=2-2/ p{u2,V2,z,y)7T{dy)-2 p{ui,vi,x,y)n{dy) 
Ja+ Ja~ 

<2 - 2(57r(yl+) - 267r{A-) 
=2-267i{A). 
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Then, together with fl4.3p . we get 

{BSi - BS2){x) - {BS, - BS2){z) < 2(1 - 6n{A))\\Si - Sail- 
Now by the definition of the quotient norm (13.51) 

\\U{BSi - BS2)\\ < (1 - Sn{A))\\Si - S2\\. 

So we obtain 

115^1 - BS2\\ < (1 - Sn{A))\\Si -S2\\. 

D 

Theorem 4.2 (On the average cost). // Iji4-i\ ) holds, then there exists a unique 
A G M and a function S* G Cb{X), unique up to equivalence (i.e. S* = 11(5'*) 
is unique), such that 

B{S*) = \ + S*. (4.4) 

Moreover, for every S G Cb{X), 

\\B^S-TX\\<\\S*\\ + \\S* -S\\ (4.5) 

hm = A (4.6) 

and, for every S G Cb{X), 

\\B^S-S*\\ < {l-67i{A)f\\S-S*\\. (4.7) 

Proof. First, by the contraction property (14. 2 p of the operator B on Cb{X) 
and the contraction principle (i.e. Banach fixed point theorem), the operator 
B has a unique fixed point in Cb{X). Namely, there exists a unique S* G 
Cb{X) such that 

BS* = S*. (4.8) 

Consequently, there exists a A G M and a function S* G Cb{X), unique up to 
equivalence such that (14. 4 p holds. 

Next, we shall prove (14. 5p . On one hand, by (14. 4p and (13. 3 p 

B^S* = B^-\S* + A) = B^~\S* + 2X) = --- = TX + S* 

and together with the triangle inequality, for any S G Cb{X), we get 

\\B^S-B^S*\\ = \\B^S-TX-S*\\ > \\B^S-TX\\-\\S*\\. (4.9) 
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On the other hand, by (13.41) . we have, 

\\B^S-B^S*\\ < \\B^-^S-B^-^S*\\ <■■■< \\S-S*\\. (4.10) 

By combining (14.91) and (14.101) . we have 

\\B^S-TX\\-\\S*\\ < \\S-S*\\ 

which clearly gives the inequality (14. 5p . 

Equation (14. 6 p is implied by (14. 5 p by passing to the limit T — ;• oo, 

^ ,. \\B'^S-TX\\ , ||5*|| + ||5-S*|| 

< hm < lim - — = 

- T^oo T ~ T->oo T 

as the functions S and 5** are all bounded functions. Equation (14. 6p . in turn, 
implies the uniqueness of A in (14. 4p . 

Further, for every 5" G Cb{X), by (14. 8 p we have 

\\B^S - S*\ =\\B{B^~^S) - i3{B^~^S*)\\ 

<{l - 57r{A))\\B^-'S - B^-^S*\\ 

<■ ■■ 

<{l-6n{A)f\\S-S*\\ 

which completes the proof. D 

Remark 4.3. The unique A in (J^) can be interpreted as the average cost 



of player 1 from the long time Markov game. 

Now we are ready for the first turnpike theorem on the set of opti- 
mal strategies. Here we introduce some notations. Denote by E{x, S) = 
{{u{x),v{x))} the set of pairs of optimal (equilibrium) strategies {u{x),v{x)) 
of dilD for any S G Cb{X). Let E{S) = U^^x{Eix, S)} be the set of pairs of 
optimal (equilibrium) strategies with final cost function S G Cb{X). In par- 
ticular, the family E{S*) contains the sets of the stationary strategies in the 
game and E{B'^~^S) is that the family of the sets of optimal (equilibrium) 
strategies at the step t of a T-step game with final cost function S G Cb{X). 

Theorem 4.4 (Early turnpikes on the set of strategies). Let ( [y^.i[ ) hold. 
Assume the set-valued function E{-) is continuous at least at the point S = 
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S* . Then for an arbitrary neighbourhood U{E{S*)) of the set E{S*), there 
exists an M & N such that if T > M, then for any S G Cb{X) 



E{B'-'S) CU{E{S*)) 



for allt<T - M. 



Remark 4.5. The continuity of the set-valued function E{-) is in the sense 
of Hausdorff metric. The property that E{-) is continuous at the point S = 
S* has been used and discussed in many places. For example, in 13^ . this 
property is called the sensitivity property for a control law. 

Proof. By (14. 7p . we have for any fixed T 

n^^S — !■ S* for large enough T — t. 
By the assumption that E{S) depends continuously on S around S'*, we have 

E{B'^-'S) -^ E{S*) for large enough T - t. 
So there exists an M G N, such that 

E{B^''S)CU{E{S*)), t<T-M 

which completes the proof. D 

Remark 4.6. This theorem states that , in a sufficiently large T-step game 
with an arbitrary final cost function S G Ci,{X), the optimal strategies at steps 
t < T — M must be close to the stationary strategies E{S*) and diverge away 
from the turnpike only near to the end of the game. Thus this is a stochastic 
competitive control version of the early turnpike as defined in [67^ . 

Further, with additional assumptions on the set of optimal strategies and 
on the equilibrium transitions, we shall show that, in a long-time game, if 
the game is carried out with optimal strategies, then, apart from short initial 
and final periods of time, the distributions of its position on the state space 
must be very close to the stationary distribution. Let || ■ Htf denote the total 
variation norm of measures. 

Theorem 4.7 (Middle turnpike on the (distribution of) state space). Let 
( 1^. j| ) hold. Assume the set-valued function E{S) is continuous at least at 
the point S = S* and, for each state x & X , the set E{x, S*) contains only 



one pair of optimal strategies {u* (x) , v* (x)) . Let Q* = {q*{x)}xex denote the 
stationary distribution for the stationary Markov chain defined on the state 
space X by the unique pair of optimal strategies {u* (x) , v* {x)) . Assume also 
that the equilibrium transition probabilities depend locally Lipschitz continu- 
ous on S around S* , i.e. that there exists a constant k such that, 

\\P{u{x),v{x),x,-)-P{u*{x),v*{x),x,-)\\tv< k\\S-S*\\, \/x e X (4.11) 

where {u, v) G E{x, S) and S is sufficiently close to S* . Then for each e > 0, 
there exists an M ^ N such that for each T-step game, T > 2M, with 
terminal cost S G Cb{X) of player 1, we have 

\\Q{t) - Q*\\tv < e 

for all t G [M,T — M], where Q{t) = {q{x,t)}xex andq{x,t) is the probability 
that the process is in a state x & X at time t if the game is carried out with 
the optimal strategies. 

Remark 4.8. In other words, q*{x) is the mean amount of time that each 
sufficiently long game with optimal strategies spends in position x. 

Proof. From fl4.1ip . together with Lemma [4.11 we have 

\\Pt{x,-) - P*{x,-)\\tv < ke^'-'WS - S*\\ 

where e = l — 6n{A), Pt{x, ■) is the transition probabihty arising from optimal 
strategies {u{x),v{x)) at time t <T with terminal payoff S and P*(x, ■) = 
P{u*{x),v*{x),x, ■). 

Let \E'i and \E'* denote linear operators on the set of measures 



(^i/i)(dx) := / iJ,{dz)Pt{z,dx) and (^*/i)(da;) := / fi{dz)P*{z,dx) 
so that 

||^i-^*||= sup n{dz){Pt{z,dx) - P*{z,dx)) 

\\^j,\\=iJ 

< ke^-'\\S - S*\\. 
The state distribution at time t can be estimated by 

g(t)=^i---^ig(o) 

=(vi/* + ei)---(v&* + ei)Q(o) 
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with ^j < ke^ ^ , where Q(0) is the initial state distribution. Since 

<fce(^-*)(l + e + e2 + ...) 



1-e - 1-e 

for t < T - Ml, Ml G N, we have 

\\Q{t) - {^*YQm\TV < ei 

for t < T — Ml, where ei can be made arbitrary small by choosing large 
enough Mi G N. 

By the theorem on the convergence of probability distribution in homo- 
geneous Markov chains to a stationary distribution, it follows that for each 
£2 > there exists an M2 G N such that 

||(^*)*Q(0)-Q*||Ty<e2, t>M2. 

Let M = max{Mi, M2}. If T > 2M, the intersection of the sets [1, T - Mi] 
and [M2, 00) is non-empty. Then we have 

\\Qit)~Q*\\Tv<ei + e2 \/te[M,T-M] 

which completes the proof. D 

5 Concluding remarks 

We have considered a two-person zero-sum Markov game with a general Borel 
state space and the modelling is formulated in the discrete time setting. With 
the Bellman operator defined by (13. 2p . we have proved the convergence of the 
long time average cost (reward) of the game and turnpike theorems on the 
set of optimal strategies and on the state space are proved. We would like 
to mention that these results still hold under the corresponding continuous 
time setting by discretising, that is by looking at a continuous time game as 
the limit of games with discrete times. 
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Notice that in our modelling, at each step, we can observe the exact posi- 
tion of this game. But it often happens that one has to take into account not 
only the exact position, but also, the state distribution (say, its variance of 
volatility). This leads to the turnpike properties for measure- valued Markov 
games, see |52l |53l El]. For example, the final cost function may be any 
continuous function on a measure space and the stochastic transition kernels 
may depend on the state distribution. We shall develop results under this 
more general setting in future work. 

References 

[1] Atsumi, H. (1965). Neoclassical Growth and the efficient program of capital 
accumulation. Review of Economic Studies, 32(2), 127-36. 

[2] Alvarez O., M. Bardi (2007). Ergodic problems in differential games. Advances 
in Dynamic Gam,e Theory, S. Jorgensen, M. Quincampoix, and T.L. Vincent, 
eds., pp 131-152, Ann. Internat. Soc. Dynam. Games, vol. 9, Birkhaeuser, 
Boston. 

[3] Alvarez O., M. Bardi (2010). Ergodic Ergodicity, stablization, and singular 
perturbations of Bellman- Isaacs equations, Memoirs American Math. Soc, 77 
pages. 

[4] Arkin V. I., I.V. Evstigneer (1987). Stochastic Models of Control and Eco- 
nomic Dynamics (Economic Theory, Econometrics, and Mathematical Eco- 
nomics). Academic Press. 

[5] Araujo, A. Scheinkman, J. A. (1977). Smoothness, comparative dynamics, and 
the turnpike property. Econometrica, 45(3), 601-620. 

[6] Bewley T.F. (1982). An integration of equilibrium theory and turnpike theory. 
Journal of mathematical economics, 20, 233-267. 

[7] Bewley, T.F. (2007). General equilibrium. Overlapping Generations Models 
and Optimal growth theory. Harvard University Press, Cambridge. 

[8] Bardi M. (2009). On differential games with long-time-average cost. Annals 
of the International Society of Dynamic Games, 10(1), 1-16. 

[9] Blot J., Crettez, B (2007). On the smoothness of optimal paths II: some local 
turnpike results. Dicisions Econ Finan, 30, 137-150. 



21 



[10] Brock, W.A., Mirman, L.J. (1972). Optimal economic growth and uncer- 
tainty: the discounted case. Journal of Economic Theory, 4, 479-513. 

[11] Brock, W.A., Mirman, L.J. (1973). Optimal economic growth and uncer- 
tainty: the no discounting case. International Economic Review, 14(3), 560- 
573. 

[12] Brock, W.A., Majumdar, M. (1978). Global asymptotic stability results for 
multisector models of optimal growth under uncertainty when future utilities 
are discounted. Journal of Economic Theory, 18, 225-243. 

[13] Cass D. (1966). Optimum Growth in an Aggregative Model of Capital Accu- 
mulation: A Turnpike Theorem. Econometrica, 34(4), 833-850. 

[14] Carlson D. A. (1990). The existence of catching-up optimal solutions for a class 
of infinite horizon optimal control problems with time delay. SI AM Journal 
on Control and Optimization, 28, 402-422. 

[15] Carlson D.A., Jabrane A., Haurie A. (1987). Existence of overtaking solutions 
to infinite dimensional control problems on unbounded time intervals. SIAM 
Journal on Control and Optimization, 25, 1517-1541. 

[16] Carlson D.A., Haurie A.B. (1996). A turnpike theory for infinite horizon open- 
loop differential games with decoupled controls. SIAM Journal on Control and 
Optimization, 34(4), 1405-1419. 

[17] Chang, F.R. (1982). A note on the stochastic value loss assumption. Journal 
of Economic Theory, 26, 164-170. 

[18] Coles, J.L. (1985). Equilibrium turnpike theory with constant returns to scale 
and possibly heterogeneous discount factors. International Economic Review, 
26(3), 671-679. 

[19] Cox J.C., C.-F. Huang (1992). A continuous-time portfolio turnpike theorem. 
Journal of Economic Dynamics and Control, 16, 491-507. 

[20] Dana, R. A. (1974). Evaluation of development programs in a stationary 
stochastic economy with bounded primary resources. In: Proceedings of the 
Warsaw Symposium on Mathematical Methods in Economics. North-Holland, 
Amsterdam, 179-205. 

[21] Dasgupta, S., Mckenzie, K.W. (1985). A note on comparative dynamics of 
stationary states. Econ. Lett, 18, 333-338. 



22 



[22] Denardo E.V., Rothbluni U.G. (2006). A turnpike theorem for a risk-sensitive 
Markov decision process with stopping. SIAM J. Control Optimal, 45(2), 414- 
431. 

[23] Dindo P. (2005). A tractable evolutionary model for the Minority Game with 
asymmetric payoffs. Physica, A (355), 110-118. 

[24] Dybvig P.H., Rogers L.C. G., Back Kerry (1999). Portfolio Turnpikes, the 
Review of Financial Studies, 12(1), 165-195. 

[25] Evstigneev I.V. (1974). Optimal stochastic programs and their stimulating 
prices, mathematical Models in Economics, J. Los and M.W. Los (eds.), 219- 
252, North-Holland. 

[26] Evstigneev I.V. (1976). Turnpike theorems in probabilistic models of economic 
dynamics. Mathematical Notes 19(2), 165-171, (translated from Matematich- 
eskie Zametki, Vol. 19(2), 279-290, 1976). 

[27] Dorfman R., P. Samuelson and R. Solow (1958). Linear programming and 
economic analysis. New York: McGraw-Hill. 

[28] Fershtman C., M. I. Kamien (1990). Turnpike properties in a finite horizon dif- 
ferential game: dynamic duopoly with sticky prices. International Economic 
Review, 31(1), 49-60. 

[29] Follmer, H., Majumdar, M. (1978). On the asymptotic behaviour of stochastic 
economic processes. Journal of Mathematical Economics, 5, 275287. 

[30] Feinstein C.D., Oren S.S. (1985). A "funnel" turnpike theorem for optimal 
growth problems with discounting. Journal of Economic Dynamics and Con- 
trol, 9, 25-39. 

[31] Fleming W., S.P. Sethi, Soner H.M. (1987). An optimal stochastic production 
planning problem with randomly fluctuating demand. 5'L4-M J. control Optim, 
25, 1494-1502. 

[32] Gale, D. (1967). On optimal development in a multi-sector economy. Rev. 
Econ. Stud., 34, 1-18. 

[33] Guo X. and O. Hernandez-Lerma (2005). Zero-sum continuous-time Markov 
games with unbounded transition and discounted payoff rates. Bernoulli, 11, 
1009-1029. 

[34] Hankansson, N. (1974). Convergence to isoelasitic utility and policy in multi- 
period portfolio choice. Journal of Financial economics, 1, 201-224. 

23 



[35] Haurie A. (1976). Optimal control on an infinite time horizon: the turnpike 
approach. Journal of mathematical economics, 3(1), 81-102. 

[36] Haurie A., Delft C.V.(1991). Turnpike properties for a class of piecewise de- 
terministic systems arising in manufacturing flow coiatro\. Annals of operations 
Research, 29, 351-374. 

[37] Hernandez-Lerma O. and J. B. Lasserre (2001). Zero-Sum Stochastic Games 
in Borel Spaces: Average Payoff Criteria, SIAM J. Control Optim., 39, 1520- 
1539. 

[38] U. D. Holzbaur. Fixed point theorems for discounted finite Markov decision 
processes. J. Math. Anal. Appl. 116:2 (1986), 594-597. 

[39] Huang M.Y., R.P. Malhame, P. E. Caines (2006). Large population stochas- 
tic dynamic games: closed-loop Mckean- Vlasov systems and the Nash cer- 
tainty equivalence principle. Communications in Information and Systems, 
6(3), 221-251. 

[40] Huang C.Fu., Zariphopoulou T. (1999). Turnpike behaviour of long-term in- 
vestments. Finance Stochastic, 3, 15-34. 

[41] Huberman G. and Ross S. (1983). Portfolio turnpike theorems, risk aversion 
and regularly varying functions. Econometrica, 51, 1345-1361. 

[42] Jaskiewicz A. (2002). Zero-sum semi-Markov games. SIAM J. Control Optim, 
41, 723-739. 

[43] Jaskiewicz A. (2009). Zero-sum ergodic semi-Markov games with weakly con- 
tinuous transition probabilities. J Optim Theory App, 141, 321-347. 

[44] Jaskiewicz A., Nowak, A.S. (2006). Zero-sum egodic stochastic games with 
Feller transition probabilities. SIAM J. control Optimal, 45, 773-789. 

[45] Joshi S. (1997). Turnpike theorems in Nonconvex nonstationary Environ- 
ments. International econmic review, 38(1), 225-248. 

[46] Joshi S. (2003). The stochastic turnpike property without uniformity in con- 
vex aggregate growth models. Journal of economic Dynamics and control, 27, 
1289-1315. 

[47] Takashi K., Roy S. (2006). Dynamic optimization with a non smooth, non- 
convex technology: the case of a linear objective function. Economic Theory, 
29, 325-340. 



24 



[48] Khan M. A. and Piazza A. (2011). An overview of turnpike theory: towards 
the discounted deterministic case. Adv. Math. Econ, 14, 39-67. 

[49] Kolokoltsov V.N. Turnpikes and infinite extremals in Markov decision pro- 
cesses. (Russian) Mat. Z ametki A6:4 (1989), 118120. 

[50] Kolokoltsov V.N. (1992). On linear, additive, and homogeneous operators in 
Idempotent Analysis. Advances in Soviet Mathematics 13, Idempotent Anal- 
ysis, Ed. V.P.Maslov et S.N. Samborski, 87-101. 

[51] Kolokoltsov V.N. and O.A. Malafeyev (2010). Understanding Game Theory: 
Introduction to the analysis of many agent systems of competition and coop- 
eration. World Scientific. 

[52] Kolokoltsov V.N. (2010). Nonlinear Markov Processes and Kinetic Equations. 
Cambridge University Press, Cambridge. 

[53] Kolokoltsov V.N. (2012). Nonlinear Markov games on a finite state space 
(mean- field and binary interactions), to appear in International Journal of 
Statistics and Probability. 

[54] Kolokoltsov V.N. , J. Li and W. Yang (2012). Mean Field Games and Non- 
linear Markov Processes. larXiv:1112.3744V 

[55] Kiienle, H.U.(2007). On Markov games with average reward criterion and 
weakly continuous transition probabilities. SIAM J. Control Optim, 45, 2156- 
2168. 

[56] Leland, H. (1972). On turnpike portfolios. In G. Szego and K. Shell, (eds.). 
Mathematical Methods in Investment and Finance. Amsterdam: North- 
holland. 

[57] Guerrero-Luchtenberg, Cesar Leonardo (2000). A uniform neighborhood turn- 
pike theorem and applications. J. Nath. Econm., 34(3), 329-357. 

[58] Le Van C, Morhaim L.(2006). On optimal growth models when the discount 
factor is near 1 or equal to 1. International Journal of economic theory, 2(1), 
55-76. 

[59] Mamedov M. A. (2002). Asymptotical stability of optimal paths in nonconvex 
problems. Research Report 02/02, University of Ballarat, February . 

[60] Mamedov, M. A. (2003). A turnpike theorem for continuous-time control sys- 
tems when the optimal stationary point is not unique. Abstract and Applied 
Analysis, 11, 631-650. 

25 



[61] Majumdar M., Mitra T. (1982). Intertemporal allocation with a Non-convex 
technology: the aggregative framework. Journal of Economic theory, 27, 101- 
136. 

[62] Marena, M., Montrucchio, L.(1999). Neighborhood turnpike theorem for con- 
tinuous time optimization models. J. Optini. Theory appL, 101, 651-676. 

[63] Majumdar and Nermuth (1982). Dynamic Optimisation in Non-Convex Mod- 
els with Irreversible investment: Monotonicity and Turnpike Theorems. Jour- 
nal of Economics, 42, 339-362 

[64] Makarov V.L. and A. M. Rubinov (1973). Mathematical Theory of Economic 
Dynamics and Equilibria, Nauka, Moscow, English trans.: Springer- Verlag, 
New York. 

[65] Maitra A. and Sudderth W. (1993). Borel stochastic games with limsup pay- 
offs. Ann. probab., 21, 861-885. 

[66] McKenzie L. W. (1963). The turnpike theorem of Morishima. Review of Eco- 
nomic Studies, 30, 169-176. 

[67] McKenzie L. W. (1976). Turnpike theory. Econometrica, 44(5), 841-865. 

[68] McKenzie L. W. (1977). A new route to the turnpike. In Henn, R. Moeachlin, 
O. (eds.) Mathematical Economics and Games Theory, Springer, New York 

[69] Mckenzie, L. W. (1979). Optimal Economic growth and turnpike theorems. 
Disussion paper 79-1, uiversity of Rochester. 

[70] McKenzie L. W. (1986). Optimal economic growth, and turnpike theorems 
and comparative dynamics, In: Handbook of Mathematical Economics, 3, 
1281-1355. 

[71] McKenzie, L.W.(1998). Turnpikes. The American Economic Review, Papers 
and Proceedings of the Hundred and Tenth Annual meeting of the American 
Economic Association. 88(2), 1-14. 

[72] Montrucchio L. (1995). A turnpike theorem for continuous-time optimal- 
control models. Journal of economic Dynamics and Control, 19, 599-619. 

[73] Montrucchio L. (1995). A new turnpike theorem for discounted programs. 
Econ. theory, 5, 371-382. 

[74] Mirman, L.J., Zilcha, I. (1975). On optimal growth under uncertainty. Journal 
of Economic Theory, 11, 329-339. 

26 



[75] Mirman, L.J., Zilcha, I. (1977). Characterizing optimal policies in a one-sector 
model of economic growth under uncertainty. Journal of Economic Theory, 
14, 389-401. 

[76] Majumdar, M., Zilcha, I. (1987). Optimal growth in a stochastic environment: 
some sensitivity and turnpike results. Journal of Economic Theory, 43, 116- 
133. 

[77] Morishima, Michio (1961). Proof of a Turnpike theorem: the no joint produc- 
tion case. Review of Economic Studies, 28, 89-97. 

[78] Mossin, J. (1968). Optimal multiperiod policies. J. Business, 31, 215-229. 

[79] Park H. (2000). Global asymptotic stability of a competitive equilibrium with 
recursive preferences. Economic theory, 15(3), 565-584. 

Radner, Roy (1961). Paths of Economic growth that are optimal with regard 
only to Final States. Review of Economic Studies, 28, 98-104. 

Alain Rapaprot and Pierre Cartigny (2004). Turnpike theorems by a value 
function approach. SAIM: control, Optimisation and Calculus of Variations, 
10, 123-141. 

Samuelson, P. A., R.M. Solow (1956). A complete capital model involving 
heterogeneous capital goods. Q. J. Econ, 27, 537-562. 

J. F. Shapiro. Turnpike Planning Horizon for a Markov Decision Model. Man- 
agement Science 14 (1968), 292-300. 

[84] Sorin S. (2005). New approaches and recent advances in two-person zero-sum 
repeated games. Advances in Dynamic Games, Nowak, Andrzej S.; Szajowski, 
Krzysztof (Eds.), 67-94, 

[85] Scheinkman J. (1976). On optimal steady states of n-sector growth models 
when utility is discounted. Journal of Economic theory, 12, 11-30. 

Vega-Amaya, O. (2003). Zero-sum semi-Markov games, fixed-point solutions 
of the Shapley equation. SIAM J. control Optm, 42, 1876-1894. 

Yano M.(1984). Competitive equilibria on turnpikes in a McKenzie Economy, 
I: A neighborhood turnpike theorem. International Economic review, 25(3) 
695-717. 

Yakovenko S.Yu., L.A. Kontorer (1992). Nonlinear semigroups and infinite 
horizon optimization. Advances in Soviet mathematics, 13,167-210. 

27 



[89] Zaslavski A.J. (1995). Optimal programs on infinite horizon 1. SIAM Journal 
on Control and Optimization, 33, 1643-1660. 

[90] Zaslavski A.J. (1995). Optimal programs on infinite horizon 2, SIAM Journal 
on Control and Optimization, 33, 1661-1686. 

[91] Zaslavski A.J. (1999). The turnpike property for dynamic discrete time zero- 
sum games. Abstract Applied Analysis, 4(1), 21-48. 

[92] Zaslavski A.J. (1998). Turnpike theorem for convex infinite dimensional. Jour- 
nal of convex analysis, 5(2), 237-248. 

[93] Zaslavski A.J. (2004). Turnpike theorem for a class of discrete time optimal 
control problem. Fixed point theory and applications, 5, 175-182. 

[94] Zaslavski A.J. (2006). Turnpike Properties in the Calculus of Variations and 
Optimal Control, Springer, New York. 

[95] Zaslavski A.J. (2007). Turnpike results for a discrete-time optimal control 
system arising in economic dynamics. Nonlinear Analysis, 67, 2024-2049. 

[96] Zaslavski A.J. (2009). Two turnpike results for a discrete-time optimal control 
system. Nonlinear Analysis, 71, 902-909. 



28 



